Estimation Of Unknown Model Parameters Research Articles

Sobolev Calibration of Imperfect Computer Models

Calibration refers to the statistical estimation of unknown model parameters in computer experiments, such that computer experiments can match underlying physical systems. This work develops a new calibration method for imperfect computer models, Sobolev calibration, which can rule out calibration parameters that generate overfitting calibrated functions. We prove that the Sobolev calibration enjoys desired theoretical properties including fast convergence rate, asymptotic normality and semiparametric efficiency. We also demonstrate an interesting property that the Sobolev calibration can bridge the gap between two influential methods: L 2 calibration and Kennedy and O’Hagan’s calibration. In addition to exploring the deterministic physical experiments, we theoretically justify that our method can transfer to the case when the physical process is indeed a Gaussian process, which follows the original idea of Kennedy and O’Hagan’s. Numerical simulations as well as a real-world example illustrate the competitive performance of the proposed method.

Integrated L1 adaptive cartesian PD control for robotic loco-manipulation

This paper presents an integrated framework of L1 adaptive control and Cartesian PD control that overcomes the inherent limitations of the latter controller in tracking end effector position for loco-manipulation tasks. In a simplified system, this work represented the loco-manipulation problem with the Cartesian space dynamics of the end effector object, capturing the dominating parts of the dynamics in linear terms. The unmodeled nonlinear dynamics, along with other model uncertainties and external disturbances, are accounted for by the L1 adaptive controller, taking the Cartesian PD input as a component of the control law to resemble its behavior for the desired system dynamics modeled in the state predictor of the L1 adaptive controller. Through simulation results of various combinations of model perturbations and external disturbances, we show that this integrated framework can achieve similar tracking performances of the end effector and object without the need to re-tune controller parameters. This work demonstrated the mitigation of effects from system uncertainties with model perturbation of up to 100% higher than expected mass and up to 15N of sinusoidal external disturbances, with higher disturbance rejection possible through designing for more conservative estimates of unknown model parameters. Combined with the low computational cost of this framework, the improvement in state command tracking gives rise to potential implementation on robotic systems controlled with linear optimization formulations.

The Data Assimilation Approach in a Multilayered Uncertainty Space

The simultaneous consideration of a numerical model and of different observations can be achieved using data-assimilation methods. In this contribution, the ensemble Kalman filter (EnKF) is applied to obtain the system-state development and also an estimation of unknown model parameters. An extension of the Kalman filter used is presented for the case of uncertain model parameters, which should not or cannot be estimated due to a lack of necessary measurements. It is shown that incorrectly assumed probability density functions for present uncertainties adversely affect the model parameter to be estimated. Therefore, the problem is embedded in a multilayered uncertainty space consisting of the stochastic space, the interval space, and the fuzzy space. Then, we propose classifying all present uncertainties into aleatory and epistemic ones. Aleatorically uncertain parameters can be used directly within the EnKF without an increase in computational costs and without the necessity of additional methods for the output evaluation. Epistemically uncertain parameters cannot be integrated into the classical EnKF procedure, so a multilayered uncertainty space is defined, leading to inevitable higher computational costs. Various possibilities for uncertainty quantification based on probability and possibility theory are shown, and the influence on the results is analyzed in an academic example. Here, uncertainties in the initial conditions are of less importance compared to uncertainties in system parameters that continuously influence the system state and the model parameter estimation. Finally, the proposed extension using a multilayered uncertainty space is applied on a multi-degree-of-freedom (MDOF) laboratory structure: a beam made of stainless steel with synthetic data or real measured data of vertical accelerations. Young’s modulus as a model parameter can be estimated in a reasonable range, independently of the measurement data generation.

Bayesian Estimation and Prediction for Inverse Power Maxwell Distribution with Applications to Tax Revenue and Health Care Data

This article discusses the problem of estimating the unknown model parameters as well as prediction of future observations from inverse power Maxwell distribution. The maximum likelihood method is applied for estimating the model parameters using Newton-Rapson iterative procedures. The existence and uniqueness of maximum likelihood estimates are established using Cauchy-Schwartz inequality. Approximate confidence intervals are constructed using Fisher information matrix. Using independent gamma informative priors, the Bayes estimates of unknown model parameters are obtained under squared error and Linex loss functions. Two approximation techniques namely: Lindley’s approximation and Metropolis-Hastings within Gibbs sampler algorithm have been employed to derive the Bayes estimators and also to construct the associate highest posterior density credible intervals. Based on the informative (observed) sample, Bayesian prediction, predictive density, and predictive intervals are derived for future observation and decision. The performance of proposed methods are evaluated though a Monte Carlo simulation experiment. Two real-life datasets related to tax revenue and heath are incorporated to show the practical utility of proposed methodology in real phenomenon.

Statistical inference under joint type-II censored data from two Burr-XII populations

The Burr-XII distribution has been widely applied in engineering, reliability, and survival analysis. Due to its importance, in this article, statistical inferences for Burr-XII distribution under a joint type-II censoring scheme is discussed. The classical likelihood estimation of unknown model parameters is studied via different calculating approaches, such as the expectation-maximization (EM) algorithm and approximate confidence intervals (ACIs) using the observed Fisher information matrix are obtained. The asymptotic bootstrap confidence intervals are also computed. In the sequel, Bayesian estimations of unknown parameters with a gamma prior distribution are considered under squared error, linear-exponential, and generalized entropy loss functions. Subsequently, we calculate the Bayesian credible interval using the importance sample. The performance of the developed methods is investigated through a Monte Carlo simulation study and two real-life examples. The results showed that the proposed estimation strategies have satisfactory results. However, Bayesian approaches were preferable to EM in terms of lower mean square error and higher coverage probability.

Modeling and Adaptive Parameter Estimation for a Piezoelectric Cantilever Beam

This paper proposes a new adaptive estimation approach to online estimate the model parameters of a piezoelectric cantilever beam. The beam behavior is firstly modeled using partial differential equations (PDE) considering the Kelvin-Voigt damping. To facilitate the estimation of unknown model parameters, the Galerkin’s method is introduced to extract desired vibration modes by separating the time and space variables of the PDE. Then, considering two major vibration modes, the corresponding system model can be represented by a fourth-order ordinary differential equation (ODE). Finally, by using measured input and output information, a novel adaptive parameter estimation strategy is introduced to estimate the unknown parameters of the derived ODE model in real time. For the purpose of driving the parameter updating law, the estimation error is extracted by using an auxiliary variable and a time-varying gain. Consequently, the convergence of the parameter estimation error is rigorously proved based on the Lyapunov theory. Simulations and experimental results show the validity and practicability of the proposed estimation method.

FAILURE RATE REGRESSION MODEL BUILDING FROM AGGREGATED DATA USING KERNELBASED MACHINE LEARNING METHODS

The problem of regression model building of equipment failure rate using datasets containing information on number of failures of recoverable systems and measurements of technological and operational factors affecting the reliability of production system is considered. This problem is important for choosing optimal strategy for preventive maintenance and restoration of elements of process equipment, which, in turn, significantly affects the efficiency of production management system. From a practical point of view, of greatest interest is the development of methods for regression models building to assess the impact of various technological and operational factors controlled during system operation on failure rate. The usual approach to regression models construction involves preselecting the model structure in the form of a parameterized functional relationship between failure rate and affecting technological variables followed by statistical estimation of unknown model parameters or training the model on datasets of measured covariates and failures.The main problem lies precisely in the choice of model structure, the complexity of which should correspond to amount of data available for training model, which in the problem of failure rate modeling is greatly complicated by lack of a priori information about its dependence on affecting variables. In this work, such a problem is solved using machine learning methods, namely, kernel ridge regression, which makes it possible to effectively approximate complex nonlinear dependences of equipment failure rate on technological factors, while there is no need to pre-select the model structure. Preliminary aggregation of data by combination of factor and cluster analysis can significantly simplify model structure. The proposed technique is illustrated by solving a practical problem of failure rate model building for semiconductor production equipment based on real data.

Comparative study of the main electromagnetic models applied to melt pool prediction with gas metal arc: Effect on flow, ripples from drop impact, and geometry

The present work concerns the electromagnetic force models in computational fluid dynamics simulations of melt pools produced with electric arcs. These are commonly applied to gas metal arcs with metal transfer, in welding and additive manufacturing. Metal drop impact on the melt pool is thus included in this study. The electromagnetic force models applied in literature use either numerical solutions of Poisson equations or one of the two analytical models developed by Kou and Sun, or Tsao and Wu. These models rely on assumptions for which the effect on the melt pool predictions remains to be understood. The present work thoroughly investigates those assumptions and their effects. It has been supported by dedicated experimental tests that did provide estimates of unknown model parameters and validation data. The obtained results show that the assumptions that fundamentally distinguish these three models change the electromagnetic force, including the relation between its components. These changes, which can also be spatially non-uniform, are large. As a result, these models lead to significantly different recirculation flow pattern, thermal convection, melt pool morphology, bead dimensions, and free surface response to the metal transfer. We conclude by proposing conditions in which each of these models is suited or questionable.

Mini-batch optimization enables training of ODE models on large-scale datasets

Quantitative dynamic models are widely used to study cellular signal processing. A critical step in modelling is the estimation of unknown model parameters from experimental data. As model sizes and datasets are steadily growing, established parameter optimization approaches for mechanistic models become computationally extremely challenging. Mini-batch optimization methods, as employed in deep learning, have better scaling properties. In this work, we adapt, apply, and benchmark mini-batch optimization for ordinary differential equation (ODE) models, thereby establishing a direct link between dynamic modelling and machine learning. On our main application example, a large-scale model of cancer signaling, we benchmark mini-batch optimization against established methods, achieving better optimization results and reducing computation by more than an order of magnitude. We expect that our work will serve as a first step towards mini-batch optimization tailored to ODE models and enable modelling of even larger and more complex systems than what is currently possible.

A new class of mechanism-equivalence-based Wiener process models for reliability analysis

It is quite common to see that a unit of a product with a higher degradation rate also presents a more prominent dispersion. This phenomenon has motivated studies on developing new degradation models. In particular, a variety of Wiener process models have been developed by correlating the drift parameter and the diffusion parameter based on the statistical features of data. However, no insightful explanations are provided for such interesting correlations. In this article, degradation mechanism equivalence is first introduced based on the acceleration factor invariant principle, and the correlation between degradation rate and variation is explained using basic principles. Then, mechanism-equivalence-based Wiener process models, including a basic model and a random-effects model, are proposed to characterize such degradation behavior of a product. Analytical solutions for both point estimation and interval estimation of unknown model parameters are obtained using the maximum likelihood estimation method and an expectation–maximization algorithm. An extension of the proposed model that is able to handle accelerated degradation tests is developed. A simulation study and two real-world applications are provided to illustrate the effectiveness of the proposed models in product reliability estimation based on degradation data.

Inference and optimum life testing plans based on Type-II progressive hybrid censored generalized exponential data

This article considers inference based on Type-II progressive hybrid censored data for a generalized exponential distribution. The maximum likelihood (ML) estimates, Bayes estimates and corresponding interval estimates of unknown model parameters are derived. Prediction estimates and prediction intervals of censored observations are obtained under one- and two-sample Bayesian framework. A Monte Carlo simulation study is undertaken to compare the proposed methods of estimation. A real data set is analyzed for illustration purpose. Finally, optimal life testing plans are obtained under cost constraints using two different optimality criteria. A computational algorithm is proposed to compute the optimal plans.

Statistical Inferences of Type-II Progressively Hybrid Censored Fuzzy Data with Rayleigh Distribution

This article presents the procedures for the estimation of the parameter of Rayleighdistribution based on Type-II progressive hybrid censored fuzzy lifetime data. Classicalas well as the Bayesian procedures for the estimation of unknown model parameters has been developed. The estimators obtained here are Maximum likelihood (ML) estimator, Method of moments (MM) estimator, Computational approach (CA) estimator and Bayes estimator. Highest posterior density (HPD) credible intervals of the unknown parameter are obtained by using Markov Chain Monte Carlo (MCMC) technique. For numerical illustration, a real data set has been considered.

An optimization-based approach to calculating neutrino flavor evolution

We assess the utility of an optimization-based data assimilation (D.A.) technique for treating the problem of nonlinear neutrino flavor transformation in core collapse supernovae. D.A. uses measurements obtained from a physical system to estimate the state variable evolution and parameter values of the associated model. Formulated as an optimization procedure, D.A. can offer an integration-blind approach to predicting model evolution, which offers an advantage for models that thwart solution via traditional numerical integration techniques. Further, D.A. performs most optimally for models whose equations of motion are nonlinearly coupled. In this exploratory work, we consider a simple steady-state model with two mono-energetic neutrino beams coherently interacting with each other and a background medium. As this model can be solved via numerical integration, we have an independent consistency check for D.A. solutions. We find that the procedure can capture key features of flavor evolution over the entire trajectory, even given measurements of neutrino flavor only at the endpoint, and with an assumed known initial flavor distribution. Further, the procedure permits an examination of the sensitivity of flavor evolution to estimates of unknown model parameters, locates degeneracies in parameter space, and can identify the specific measurements required to break those degeneracies.

Bayesian inference for the parameters of Weibull distribution under progressive Type-I interval censored data with beta-binomial removals

ABSTRACTThis article considers the problem of estimating the parameters of Weibull distribution under progressive Type-I interval censoring scheme with beta-binomial removals. Classical as well as the Bayesian procedures for the estimation of unknown model parameters have been developed. The Bayes estimators are obtained under SELF and GELF using MCMC technique. The performance of the estimators, has been discussed in terms of their MSEs. Further, expression for the expected number of total failures has been obtained. A real dataset of the survival times for patients with plasma cell myeloma is used to illustrate the suitability of the proposed methodology.

Nonlinear general path models for degradation data with dynamic covariates

Degradation data have been widely used to estimate product reliability. Because of technology advancement, time‐varying usage and environmental variables, which are called dynamic covariates, can be easily recorded nowadays, in addition to the traditional degradation measurements. The use of dynamic covariates is appealing because they have the potential to explain more variability in degradation paths. We propose a class of general path models to incorporate dynamic covariates for modeling of degradation paths. Physically motivated nonlinear functions are used to describe the degradation paths, and random effects are used to describe unit‐to‐unit variability. The covariate effects are modeled by shape‐restricted splines. The estimation of unknown model parameters is challenging because of the involvement of nonlinear relationships, random effects, and shaped‐restricted splines. We develop an efficient procedure for parameter estimations. The performance of the proposed method is evaluated by simulations. An outdoor coating weathering dataset is used to illustrate the proposed method. Copyright © 2015 John Wiley & Sons, Ltd.

Empirical Bayes Approach for Developing Hierarchical Probabilistic Predictive Models and Its Application to the Seismic Reliability Analysis of FRP-Retrofitted RC Bridges

AbstractThis paper proposes a general formulation to develop hierarchical probabilistic predictive models for clustered data. The common clustering factor, shared among the data within a group, causes statistical dependence that needs to be accounted for in the estimation of unknown model parameters. The basic idea of the hierarchical formulation is that the unknown model parameters are endowed with distributions that depend on a set of shared underlying parameters, and this construction is recursive up to the highest level of the hierarchy. The usual improper noninformative prior distributions on variance parameters of hierarchical models can lead to nonexistent posterior distributions that may appear perfectly reasonable in numerical simulations. On the other hand, common proper noninformative prior distributions may also substantially affect posterior statistics. Instead, the empirical Bayes approach is proposed to objectively estimate the variance parameters. The Gibbs sampling algorithm is used to es...

Slope estimation in noisy piecewise linear functions

This paper discusses the development of a slope estimation algorithm called MAPSlope for piecewise linear data that is corrupted by Gaussian noise. The number and locations of slope change points (also known as breakpoints) are assumed to be unknown a priori though it is assumed that the possible range of slope values lies within known bounds. A stochastic hidden Markov model that is general enough to encompass real world sources of piecewise linear data is used to model the transitions between slope values and the problem of slope estimation is addressed using a Bayesian maximum a posteriori approach. The set of possible slope values is discretized, enabling the design of a dynamic programming algorithm for posterior density maximization. Numerical simulations are used to justify choice of a reasonable number of quantization levels and also to analyze mean squared error performance of the proposed algorithm. An alternating maximization algorithm is proposed for estimation of unknown model parameters and a convergence result for the method is provided. Finally, results using data from political science, finance and medical imaging applications are presented to demonstrate the practical utility of this procedure.

Lessons Learned from Quantitative Dynamical Modeling in Systems Biology

Due to the high complexity of biological data it is difficult to disentangle cellular processes relying only on intuitive interpretation of measurements. A Systems Biology approach that combines quantitative experimental data with dynamic mathematical modeling promises to yield deeper insights into these processes. Nevertheless, with growing complexity and increasing amount of quantitative experimental data, building realistic and reliable mathematical models can become a challenging task: the quality of experimental data has to be assessed objectively, unknown model parameters need to be estimated from the experimental data, and numerical calculations need to be precise and efficient.Here, we discuss, compare and characterize the performance of computational methods throughout the process of quantitative dynamic modeling using two previously established examples, for which quantitative, dose- and time-resolved experimental data are available. In particular, we present an approach that allows to determine the quality of experimental data in an efficient, objective and automated manner. Using this approach data generated by different measurement techniques and even in single replicates can be reliably used for mathematical modeling. For the estimation of unknown model parameters, the performance of different optimization algorithms was compared systematically. Our results show that deterministic derivative-based optimization employing the sensitivity equations in combination with a multi-start strategy based on latin hypercube sampling outperforms the other methods by orders of magnitude in accuracy and speed. Finally, we investigated transformations that yield a more efficient parameterization of the model and therefore lead to a further enhancement in optimization performance. We provide a freely available open source software package that implements the algorithms and examples compared here.

Integrated direct/indirect adaptive robust motion trajectory tracking control of pneumatic cylinders

This paper studies the precision motion trajectory tracking control of a pneumatic cylinder driven by a proportional-directional control valve. An integrated direct/indirect adaptive robust controller is proposed. The controller employs a physical model based indirect-type parameter estimation to obtain reliable estimates of unknown model parameters, and utilises a robust control method with dynamic compensation type fast adaptation to attenuate the effects of parameter estimation errors, unmodelled dynamics and disturbances. Due to the use of projection mapping, the robust control law and the parameter adaption algorithm can be designed separately. Since the system model uncertainties are unmatched, the recursive backstepping technology is adopted to design the robust control law. Extensive comparative experimental results are presented to illustrate the effectiveness of the proposed controller and its performance robustness to parameter variations and sudden disturbances.

Self-tuning weighted measurement fusion Kalman filtering algorithm

For the multisensor linear discrete system with correlated noises and same measurement matrix, the self-tuning weighted measurement fusion Kalman filtering algorithm is presented when the model parameters and noise variances are all unknown. It can handle the self-tuning fused Kalman filtering, smoothing, and prediction problem and the input white noise deconvolution estimation problem. By the dynamic variance error system analysis (DVESA) method, it is proved that the solution of the self-tuning Riccati equation converges to the solution of the steady-state Riccati equation. Based on the convergence of the self-tuning Riccati equation, the convergence of the proposed self-tuning weighted measurement fusion Kalman estimator is proved. So it has asymptotic global optimality. Applying to the multi-channel autoregressive moving average (ARMA) signal with sensor bias, the corresponding self-tuning weighted measurement fusion Kalman estimator of the signal is also presented, where the estimates of unknown model parameters and noise variances are obtained by the multi-dimension recursive extended least squares (RELS) algorithm, the correlation method and the Gevers–Wouters algorithm with a dead band. One simulation example shows the effectiveness.